Serving ML Predictions in batch and real-time
Overview
Duration is 1 min
In this lab, you run Dataflow pipelines to serve predictions for batch requests as well as streaming in real-time.
What you learn
In this lab, you write code to:
-
Create a prediction service that calls your trained model deployed in Cloud to serve predictions
-
Run a Dataflow job to have the prediction service read in batches from a CSV file and serve predictions
-
Run a streaming Dataflow pipeline to read requests real-time from Cloud Pub/Sub and write predictions into a BigQuery table
Setup
For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.
-
Make sure you signed into Qwiklabs using an incognito window.
-
Note the lab's access time (for example,
and make sure you can finish in that time block.
-
When ready, click
.
-
Note your lab credentials. You will use them to sign in to the Google Cloud Console.
-
Click Open Google Console.
-
Click Use another account and copy/paste credentials for this lab into the prompts.
- Accept the terms and skip the recovery resource page.
Activate Cloud Shell
Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.
In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.
Click Continue.
It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:
gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.
You can list the active account name with this command:
gcloud auth list
(Output)
Credentialed accounts:
- <myaccount>@<mydomain>.com (active)
(Example output)
Credentialed accounts:
- google1623327_student@qwiklabs.net
You can list the project ID with this command:
gcloud config list project
(Output)
[core]
project = <project_ID>
(Example output)
[core]
project = qwiklabs-gcp-44776a13dea667a6
Check project permissions
Before you begin your work on Google Cloud, you need to ensure that your project has the correct permissions within Identity and Access Management (IAM).
-
In the Google Cloud console, on the Navigation menu (
), click IAM & Admin > IAM.
-
Confirm that the default compute Service Account
{project-number}-compute@developer.gserviceaccount.comis present and has theeditorrole assigned. The account prefix is the project number, which you can find on Navigation menu > Home.
If the account is not present in IAM or does not have the editor role, follow the steps below to assign the required role.
-
In the Google Cloud console, on the Navigation menu, click Home.
-
Copy the project number (e.g.
729328892908). -
On the Navigation menu, click IAM & Admin > IAM.
-
At the top of the IAM page, click Add.
-
For New members, type:
{project-number}-compute@developer.gserviceaccount.com
Replace {project-number} with your project number.
- For Role, select Project (or Basic) > Editor. Click Save.
Creating the virtual environment
Execute the following command to download and update the packages list.
sudo apt-get update
Python virtual environments are used to isolate package installation from the system.
sudo apt-get install virtualenv
Y and then Enter.virtualenv -p python3 venv
Activate the virtual environment.
source venv/bin/activate
Copy trained model
Step 1
Set necessary variables and create a bucket:
REGION=us-central1
BUCKET=$(gcloud config get-value project)
TFVERSION=2.1
gsutil mb -l ${REGION} gs://${BUCKET}
Step 2
Copy trained model into your bucket:
gsutil -m cp -R gs://cloud-training-demos/babyweight/trained_model gs://${BUCKET}/babyweight
Deploy trained model
Step 1
Set necessary variables:
MODEL_NAME=babyweight
MODEL_VERSION=ml_on_gcp
MODEL_LOCATION=$(gsutil ls gs://${BUCKET}/babyweight/export/exporter/ | tail -1)
Step 2
Set the region to global by executing the following command:
gcloud config set ai_platform/region global
Step 3
Deploy trained model:
gcloud ai-platform models create ${MODEL_NAME} --regions $REGION
gcloud ai-platform versions create ${MODEL_VERSION} --model ${MODEL_NAME} --origin ${MODEL_LOCATION} --runtime-version $TFVERSION
Browse lab files
Duration is 5 min
Step 1
Clone the course repository:
cd ~
git clone https://github.com/GoogleCloudPlatform/training-data-analyst
Step 2
In Cloud Shell, navigate to the folder containing the code for this lab:
cd ~/training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving
Step 3
Run the what_to_fix.sh script to see a list of items you need to add/modify to existing code to run your app:
./what_to_fix.sh
As a result of this, you will see a list of filenames and lines within those files marked with TODO. These are the lines where you have to add/modify code. For this lab, you will focus on #TODO items for .java files only, namely BabyweightMLService.java : which is your prediction service.
How the code is organized
Prediction service
In this section, you fix the code in BabyweightMLService.java and test it with the run_once.sh script that is provided. If you need help with the code, look at the next section that provides hints on how to fix code in BabyweightMLService.java.
Step 1
You may use the Cloud Shell code editor to view and edit the contents of these files.
In Cloud Shell, click the Open Editor icon on the top right.
Step 2
After it is launched, navigate to the following directory:
training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving/pipeline/src/main/java/com/google/cloud/training/mlongcp
Step 3
Open the BabyweightMLService.java files and replace #TODOs in the code.
Step 4
Once completed, go into your Cloud Shell and run the run_once.sh script to test your ML service.
cd ~/training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving
./run_once.sh
Serve predictions for batch requests
This section of the lab calls AddPrediction.java that takes a batch input (one big CSV), calls the prediction service to generate baby weight predictions and writes them into local files (multiple CSVs).
Step 1
In your Cloud Shell code editor, open the AddPrediction.java file available in the following directory:
training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving/pipeline/src/main/java/com/google/cloud/training/mlongcp
Step 2
Look through the code and notice how, based on input argument, it decides to set up a batch or streaming pipeline, and creates the appropriate TextInputOutput or PubSubBigQuery io object respectively to handle the reading and writing.
Step 3
Test batch mode by running the run_ontext.sh script provided in the lab directory:
cd ~/training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
./run_ontext.sh
Serve predictions real-time with a streaming pipeline
In this section of the lab, you will launch a streaming pipeline with Dataflow, which will accept incoming information from Cloud Pub/Sub, use the info to call the prediction service to get baby weight predictions, and finally write that info into a BigQuery table.
Step 1
On your GCP Console's left-side menu, go into Pub/Sub and click the CREATE TOPIC button on top. Create a topic called babies.
Step 2
Back in your Cloud Shell, modify the script run_dataflow.sh to get Project ID (using --project) from command line arguments, and then run as follows:
cd ~/training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving
./run_dataflow.sh
This will create a streaming Dataflow pipeline.
Step 3
Back in your GCP Console, use the left-side menu to go into Dataflow and verify that the streaming job is created.
Step 4
Next, click on the job name to view the pipeline graph. Click on the pipeline steps (boxes) and look at the run details (like system lag, elements added, etc.) of that step on the right side.
This means that your pipeline is running and waiting for input. Let's provide input through the Pub/Sub topic.
Step 5
Copy some lines from your example.csv.gz:
cd ~/training-data-analyst/courses/machine_learning/deepdive/06_structured/labs/serving
zcat exampledata.csv.gz
Step 6
On your GCP Console, go back into Pub/Sub, click on the babies topic, and then click on Publish message button on the top. In the message box, paste the lines you just copied from exampledata.csv.gz and click on Publish button.
Step 7
You may go back into Dataflow jobs on your GCP Console, click on your job and see how the run details have changed for the steps, for example click on write_toBQ and look at Elements added.
Step 8
Lets verify that the predicted weights have been recorded into the BigQuery table.
Open BigQuery Console
In the Google Cloud Console, select Navigation menu > BigQuery:
The Welcome to BigQuery in the Cloud Console message box opens. This message box provides a link to the quickstart guide and lists UI updates.
Click Done.
The BigQuery console opens.
Look at the Explorer tab and expand your GCP Project ID by clicking the down arrow next to it. You should see the babyweight dataset. Click on the down arrow to its left, and you should see your predictions table.
Step 9
Type the query in the query EDITOR to retrieve rows from your predictions table.
SELECT * FROM babyweight.predictions LIMIT 1000
Step 10
Click the Run button. Notice the predicted_weights_pounds column in the result.
Step 11
Remember that your pipeline is still running. You can publish additional messages from your example.csv.gz and verify new rows added to your predictions table. Once you are satisfied, you may stop the Dataflow pipeline by going into your Dataflow Jobs page, and click the Stop button on the top. Select Drain and click Stop Job.
End your lab
When you have completed your lab, click End Lab. Qwiklabs removes the resources you’ve used and cleans the account for you.
You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.
The number of stars indicates the following:
- 1 star = Very dissatisfied
- 2 stars = Dissatisfied
- 3 stars = Neutral
- 4 stars = Satisfied
- 5 stars = Very satisfied
You can close the dialog box if you don't want to provide feedback.
For feedback, suggestions, or corrections, please use the Support tab.
©2020 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.